fix(bench): give sqs its own FIFO corpus alongside the drain deviation - #173
Merged
Conversation
The drain deviation never reaches consume_fifo: it holds no barrier and takes no drain, so it publishes the tier's 5 000 messages per shard and consumes them through the sequenced path, which LocalStack serves at a few messages per second. Measured on 2026-09-08, the 64 B FIFO cell had drained half its corpus after five hours, and there are three such cells. Add --fifo-messages to the harness, the per-shard FIFO corpus in place of the tier's per-consumer count, leaving every other flow on the tier's count; bench.sh appends it for sqs only at 100 per shard and names both deviations in its deviation line. The row records the corpus it ran in messages, as the drain rows record theirs. The README's deviation section covers it.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…rker Kafka's wrapper reports one FIFO worker and the tier already sizes its corpus for that one worker, so an override described as per shard would disagree with the count it replaces. Say per worker, which is what the code did, and pin it with a one-worker test.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
shove-docs | 6d3ecaa | Commit Preview URL Branch Preview URL |
Sep 08 2026, 03:23 PM |
zannis
added a commit
that referenced
this pull request
Sep 8, 2026
Measure SQS against LocalStack on the same host as the other published runs, with both SQS deviations in place: the 60 000-message drain corpus from #163 and the 100-per-worker FIFO corpus from #173, plus the SNS byte-limit fix from #171 so the 64 KiB batch publishes go through. 60 rows, no failures; consumer_group and broadcast are recorded as unsupported, as the capability traits say. The document now holds every backend. The README and performance page say so, and say what the SQS rows measure: LocalStack, at a recorded corpus, not the AWS service.
zannis
added a commit
that referenced
this pull request
Sep 8, 2026
* bench(results): add the sqs run to the results document Measure SQS against LocalStack on the same host as the other published runs, with both SQS deviations in place: the 60 000-message drain corpus from #163 and the 100-per-worker FIFO corpus from #173, plus the SNS byte-limit fix from #171 so the 64 KiB batch publishes go through. 60 rows, no failures; consumer_group and broadcast are recorded as unsupported, as the capability traits say. The document now holds every backend. The README and performance page say so, and say what the SQS rows measure: LocalStack, at a recorded corpus, not the AWS service. * test(chartgen): assert the grown canvas's plot minimum from the y axis, not the ticks Tick labels stop short of the plot boundaries, so a tick span at 0.8 of the minimum could pass an undersized plot. The y axis is a two-point vertical polyline and bounds the plot body exactly; assert the full minimum from it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#163's 60,000-message drain corpus never reaches
consume_fifo: the FIFO cell holds no barrier and takes no drain, so it publishes the tier's 5,000 messages per shard and consumes them through the sequenced path, which LocalStack serves at a few messages per second. Measured today, the 64 B FIFO cell had drained half its 40,000-message corpus after five hours, and the matrix has three such cells (one per payload). The SQS pass was headed for about 30 hours of FIFO.Change
--fifo-messages <n>, the per-shard corpus for theconsume-fifocell in place of the tier's per-consumer count. Every other flow keeps the tier's count. The row records the corpus it ran inmessages, as drain rows record theirs.bench.sh:SQS_FIFO_MESSAGES=100, appended forsqsonly (the matrix carries no--fifo-messages, since every other backend runs the tier's FIFO corpus), and thedeviation:line names both deviations.benches/README.md: the SQS deviation section covers the FIFO corpus.Tests
the_fifo_corpus_can_be_deviated_per_shard: with the knob, the FIFO scenario's corpus is 100 × 8 shards; the same build of the parallel flow with and without the knob yields identical corpora. Failed first on the unknown argument. Full harness suite passes (142).cargo fmt -- --checkand both clippy gates pass.